Much recent work in realistic image synthesis has focused on the use of actual data measurements of real-world surfaces and materials, both in the search for better data-driven reflectance models, and for direct use in image-based rendering techniques.
The reflectance properties of a surface can be characterized by its Bidirectional Reflectance Distribution Function (BRDF) [NICODEMUS, F. E., RICHMOND, J. C., AND HSIA, J. J. 1977. Geometric Considerations and Nomenclature for Reflectance, U.S. Dept. of Commerce, National Bureau of Standards, October 1977, incorporated by reference herein], the four dimensional function that describes how much light from any incident direction (θi,φi) is transferred to any exitant direction (θe,φe):BRDF(θi,φi,θe,φe)
The field is quite mature in techniques for measuring BRDFs, and for representing them accurately and compactly. Real world surfaces, however, are not perfectly homogeneous—they exhibit local variations in microgeometry and in reflectance, which are not adequately represented by a single BRDF.
Dana et al. define the Bidirectional Texture Function (BTF) as the six dimensional function which extends the BRDF by allowing reflectance to vary spatially along the surface, parameterized by (u,v) DANA, K. J., GINNEKEN, B. VAN, NAYAR, S. K., AND KOENDERINK, J. J. 1999. Reflectance and Texture of Real World Surfaces. ACM Transactions on Graphics, 18, 1, 1-34, incorporated by reference herein:BRDF(u,v,θi,φi,θe,φe)
This representation is able to effectively capture the various subtleties of complexly textured surfaces, particularly those exhibiting such phenomena as self-occlusion and self-shadowing.
There have been recent advances in working with BTFs for realistic image synthesis. Because the BTF is a large unwieldy 6D function, it is difficult to obtain a dense sampling, and therefore current databases are relatively sparse. Yet recent successful research has shown that even a sparse sampling of the BTF can be adequate for rendering applications. LIU, X., YU, Y., AND SHUM, H. Y. 2001. Synthesizing Bidirectional Texture Functions for Real-World Surfaces. In Proceedings of ACM SIGGRAPH 2001, ACM Press/ACM SIGGRAPH, New York. E. Fiume, Ed., Computer Graphics Proceedings, Annual Conference Series, ACM, 97-106; TONG, X., ZHANG, J., LIU, L., WANG, X., GUO, B., AND SHUM, H. Y. 2002. Synthesis of Bidirectional Texture Functions on Arbitrary Surfaces. ACM Transactions on Graphics, 21, 3, 665-672; VASILESC, M. A. O., AND TERZOPOULOS, D. 2003. TensorTextures. ACM SIGGRAPH 2003 Conference Abstracts and Applications, July 2003, all of which are incorporated by reference herein.
Increased quality of BTF sample data would also be of benefit to computer vision research. For example, algorithms that reconstruct geometry or motion from multiple views require correspondences to be found between these views. BTF data would allow robust testing of the identification of corresponding surface points, even as the appearance of each surface point varies with view angle. This data would also benefit shape-from-texture, texture segmentation, and texture recognition techniques.
Use of real-world reflectance is currently characterized by the difficulty of gathering the BRDF and the BTF, particularly due to the high dimensionality of this data.
The straightforward approach to measuring the 4D BRDF is to mechanically position a light source and photometer around the hemisphere about the sample though the use of robotic armatures, as in Murray-Coleman and Smith. MURRAY-COLEMAN, J. F., AND SMITH, A. M. 1990. The Automated Measurement of BRDFs and their Application to Luminaire Modeling. Journal of the Illuminating Engineering Society, pp. 87-99, Winter 1990, incorporated by reference herein. Any such mechanical arrangement must have four degrees of freedom; data collection is tediously performed by sequentially stepping through each position.
Subsequent methods greatly improve the efficiency of data acquisition by reducing the number of mechanically scanned dimensions through the use of a 2D imaging element such as a CCD camera. Ward's LBL imaging gonioreflectometer uses a hemi-ellipsoidal mirror. WARD, G. J. 1992. Measuring and Modeling Anisotropic Reflection. In Computer Graphics (Proceedings of ACM SIGGRAPH 92), 26, 2, ACM, 255-263, incorporated by reference herein. A CCD camera equipped with a wide-angle-lens, and the surface sample are positioned at the mirror's two respective foci to effectively map pixel position to exitant angular position. This method requires mechanical repositioning of the light source. Also notable about Ward's device is that the mirror is semi-transparent, thereby permitting measurements when view and illumination angles are coincident. Others have thoroughly explored the various other possible arrangements of curved mirrors and beam splitters. DAVIS, K. J., AND RAWLINGS, D. C. 1997. Directional reflectometer for measuring optical bidirectional reflectance. U.S. Pat. No. 5,637,873, June 1997; MATTISON, P. R., DOMBROWSKI, M. S., LORENZ, J., DAVIS, K., MANN, H., JOHNSON, P., AND FOOS, B. 1998. Hand-held directional reflectometer: an angular imaging device to measure BRDF and HDR in real-time. In Proceedings of SPIE, The International Society for Optical Engineering, Scattering and Surface Roughness II, 3426:240-251, July 1998; and CARTER, R. R., AND PLESKOT, L. K. 1999. Imaging scatterometer. U.S. Pat. No. 5,912,741, June 1999, all of which are incorporated by reference herein.
An alternative way to utilize an imaging element is to measure the BRDF on a curved sample. Lu et al. arranges a sample patch onto a known cylinder. LU, R., KOENDERINK, J. J., AND KAPPERS, A. M. L. 1998. Optical properties (bidirectional reflectance distribution functions) of velvet. Applied Optics, 37, 25, 5974-5984, incorporated by reference herein. Marschner et al. relaxes the sample geometry restriction by utilizing a range scanner, and improves acquisition flexibility by allowing for free positioning of the capture camera. MARSCHNER, S. R., WESTIN, S. H., LAFORTUNE, E. P. F., TORRANCE, K. E., AND GREENBERG, D. P. 1999. Image-based BRDF Measurement Including Human Skin. In Proceedings of the 10th Eurographics Workshop on Rendering, pp. 131-144, June 1999, incorporated by reference herein.
More recent work attempts to recover the BRDF from sampling environments that are even less structured. Boivin and Gagalowicz demonstrate recovering multiple BRDFs from a single photograph, with known geometry and light source positions. BOIVIN, S. AND GAGALOWICZ, A. 2001. Image-Based Rendering of Diffuse, Specular and Glossy Surfaces from a Single Image. In Proceedings of ACM SIGGRAPH 2001, ACM Press/ACM SIGGRAPH, New York. E. Fiume, Ed., Computer Graphics Proceedings, Annual Conference Series, ACM, 107-116, incorporated by reference herein. Ramamoorth and Hanrahan describe a signal processing framework that generalizes the recovery of the BRDF under unknown lighting conditions. RAMAMOORTHI, R. ANDHANRAHAN, P. 2001. A Signal-Processing Framework for Inverse Rendering. In Proceedings of ACM SIGGRAPH 2001, ACM Press/ACM SIGGRAPH, New York. E. Fiume, Ed., Computer Graphics Proceedings, Annual Conference Series, ACM, 117-128, incorporated by reference herein.
The seminal work by Dana et al. on the BTF [1999] presents a 3DOF robotic system that incrementally tilts/rotates a patch of the sample in front of a light source. This method produces 205 total samples of the BTF, with a relatively even distribution of illumination directions, but, due to mechanical limitations, with a limited distribution of viewing angles. It also requires a sample patch of the surface to be affixed to the device, which makes in situ measurements impossible, particularly for skin.
Other research involving BTFs utilizes various other custom gantry rigs, such as that of Furukawa et al., which uses 2 motorized concentric arcs carrying 6 cameras and 6 lights. FURUKAWA, R., KAWASAKI, H., IKEUCHI, K., AND SAKAUCHI, M. 2002. Appearance based object modeling using texture database: Acquisition, compression and rendering. In Proceedings of the 13th Eurographics Workshop on Rendering Techniques, pp. 257-266, 2002, incorporated by reference herein.
Later work by Dana introduces a BTF measurement device that utilizes a concave paraboloid mirror section, similar to that used in previous BRDF capture devices, but in concert with an aperture and a translation stage for the sample. DANA, K. J. 2001. BRDF/BTF Measurement Device. In Proceedings of Eighth IEEE International Conference on Computer Vision (ICCV), IEEE Computer Society, vol. 2, pp. 460-6, Vancouver, British Columbia, July 2001, incorporated by reference herein. Theoretically, this technique should be able to produce very high resolution sampling of the BTF in every dimension, with large flexibility in sample distribution, but at a slow capture rate. It also inherits the problems associated with the need to affix surface samples.
Note that this technique is representative of a general class of solutions to the BTF capture problem, which utilize a 4D BRDF measurement device, mechanically scanning the sample across the device to obtain the additional two dimensions.
Other techniques measure that subset of the BTF for which the viewpoint is fixed, and only illumination is varied.
Debevec et al.'s “Light Stage”, constructed to capture the complex reflectance of the human face, mechanically scans a directional light source at relatively high speeds through two degrees of freedom, capturing 64×32 illumination samples. DEBEVEC, P., HAWKINS, T., TCHOU, C., DUIKER, H. P., SAROKIN, W., AND SAGAR, M. 2000. Acquiring the Reflectance Field of a Human Face. In Proceedings of ACM SIGGRAPH 2000, ACM Press/ACM SIGGRAPH, New York. Computer Graphics Proceedings, Annual Conference Series, ACM, 145-156, incorporated by reference herein. Successive versions of the stage have replaced this single light source, first with a linear array of xenon strobes on a motorized arc, and then with a static 2D array of 156 LED clusters, allowing for the capture of subjects in motion under arbitrary illumination conditions. DEBEVEC, P., WENGER, A., TCHOU, C., GARDNER, A., WAESE, J., AND HAWKINS, T. 2002. A Lighting Reproduction Approach to Live-Action Compositing. ACM Transactions on Graphics, 21, 3, 547-556, incorporated by reference herein.
Malzbender et al. describes a device for in situ surface reflectance measurement, wherein 50 inward-pointing light sources are distributed on a small, portable hemispherical frame, allowing for rapid automated acquisition. MALZBENDER, T., GELB, D., AND WOLTERS, H. 2001. Polynomial Texture Maps. In Proceedings of ACM SIGGRAPH 2001, ACM Press/ACM SIGGRAPH, New York. E. Fiume, Ed., Computer Graphics Proceedings, Annual Conference Series, ACM, 519-528, incorporated by reference herein. Polynomial curves are fitted to the lighting-dependent color at each pixel; these curves are used to generate images with novel lighting conditions that interpolate the light positions that were sampled.
The reflectance field [Debevec 2000], is an eight dimensional function that completely describes the geometric relationship between a ray of light and a textured surface: two dimensions to describe the ray's angle of incidence, two for the exitant angle, two for the position of the surface point on the texture surface, and two for the positional shift of the ray between entering and leaving the surface due to subsurface scattering.
The term Bidirectional Reflectance Distribution Function (BRDF) was first coined by Nicodemus [Nicodemus 1977] to describe the four dimensional relationship between incident and exident light at a surface point.
Dana extended this to six dimensions, coining the term Bidirectional Texture Function (BTF) to account for the two additional dimensions of spatial variance across a texture surface [Dana et al. 1997].
Jensen pointed out that light can enter and exit at different points due to subsurface scattering [Jensen 2001].
Matusik devised a method of measuring and rerendering the analog of the six dimensional BTF for solid objects [Matusik et al. 2002].
Masselus keeps the camera fixed, while allowing lighting direction to vary. This work also used projectors as space-varying light sources for measurement [Masselus et al. 2003] as did [Han and Perlin 2003].
Levoy has also noted that an image can be used as a angularly variant light source for surface reflectance measurement [Levoy 2000].